Papers with learning strategy
Cartography Active Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to label data are limited in their notion of informativeness, due to post-training model uncertainty and batch diversity. |
| Approach: | They propose a new Active Learning algorithm that exploits the behavior of the model on individual instances during training as a proxy to find the most informative instances for labeling. |
| Outcome: | The proposed method is competitive to other common AL methods, showing that training dynamics derived from small seed data can be successfully used for AL. |
CoinMath: Harnessing the Power of Coding Instruction for Math LLM (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown strong performance in solving mathematical problems, with code-based solutions proving particularly effective. |
| Approach: | They propose a learning strategy to enhance mathematical reasoning by diversifying the coding styles of code-based rationales. |
| Outcome: | The proposed learning strategy outperforms its baseline model, MAmmoTH, which uses code-based solutions. |
CoLLaVO: Crayon Large Language and Vision mOdel (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) and instruction tuning have been used to drive the evolution of Vision Language Model (VLM) towards a versatile general-purpose model. |
| Approach: | They propose a learning strategy of Dual QLoRA to preserve object-level image understanding without forgetting it during visual instruction tuning, thereby achieving a significant leap in numerous VL benchmarks in a zero-shot setting. |
| Outcome: | The proposed model outperforms closed-source models on vision language tasks and achieves a significant leap in numerous benchmarks. |
Generalization in Instruction Following Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding and executing natural language instructions in a grounded domain is one of the hallmarks of artificial intelligence. |
| Approach: | They propose a learning strategy that involves data augmentation to improve the model's performance. |
| Outcome: | The proposed learning strategy outperforms state-of-the-art models in the blocks world domain while satisfying our expectations much better. |
PMAES: Prompt-mapping Contrastive Learning for Cross-prompt Automated Essay Scoring (2023.acl-long)
Copied to clipboard
| Challenge: | Current cross-prompt automated essay scoring systems are limited by their ability to extract features directly from the original prompt. |
| Approach: | They propose a method to learn more shared features between the source and target prompts by using a "prompt-mapping" approach to obtain more shared feature representations between the two prompts . |
| Outcome: | The proposed method can be applied to a ASAP++ dataset showing that it is highly efficient and consistent. |
Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language encoders do not precisely align words and phrases across languages. |
| Approach: | They propose a learning strategy for training robust models by drawing connections between adversarial examples and failure cases of zero-shot cross-lingual transfer. |
| Outcome: | The proposed model can achieve good performance even if representations of different languages are not aligned well. |
A Framework for Vision-Language Warm-up Tasks in Multimodal Dialogue Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for building multimodal open-domain dialogue agents based on large datasets are limited in real-world settings . |
| Approach: | They propose a new learning strategy called vision-language warm-up tasks for multimodal dialogue models that relies solely on learning from target data. |
| Outcome: | The proposed learning strategy achieves comparable and in some cases superior performance compared to existing state-of-the-art models on various evaluation metrics. |
EmByte: Decomposition and Compression Learning for Small yet Private NLP (2025.findings-emnlp)
Copied to clipboard
| Challenge: | EMBYTE is a byte-level tokenization model that reduces embedding parameters by up to 94% . it is also resilient to privacy threats such as gradient inversion attacks . |
| Approach: | EMBYTE is a byte-level tokenization model that decomposes subwords into fine-grained byte embeddings and then compresses them via neural projection. |
| Outcome: | EMBYTE achieves substantial embedding compression while preserving accuracy and enhancing privacy. |
TLDR: Extreme Summarization of Scientific Documents (2020.findings-emnlp)
Copied to clipboard
| Challenge: | TLDR generation requires expert background knowledge and understanding of complex domain-specific language. |
| Approach: | They propose a learning strategy that exploits titles as an auxiliary training signal. |
| Outcome: | The proposed method improves upon strong baselines under both automated metrics and human evaluations. |
Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task (2022.findings-emnlp)
Copied to clipboard
| Challenge: | ‘Actions’ play a vital role in how humans interact with the world and therefore require the capability to perform ‘Reasoning about Actions & Change’ (RAC). |
| Approach: | They propose an encoder-decoder architecture to learn the representation of actions as vectors and combine it with existing modality parsers and a scene graph question answering model to evaluate the proposed system. |
| Outcome: | The proposed system performs better than baselines on the CLEVR_HYP dataset and compares with existing parsers and a scene graph question answering model. |
Sentiment Interpretable Logic Tensor Network for Aspect-Term Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Aspect-term sentiment analysis (ATSA) is a fine-grained task that aims to infer the sentiment towards the given aspect-terms. |
| Approach: | They propose a novel ATSA method that is interpretable and has high accuracy . they propose SILTN, which is a neurosymbolic formalism, to improve the accuracy based on syntax knowledge distillation. |
| Outcome: | The proposed method is interpretable because it is a neurosymbolic formalism and a computational model that supports learning and reasoning about data with a differentiable first-order logic language. |
Generalizing over Long Tail Concepts for Medical Term Normalization (2022.emnlp-main)
Copied to clipboard
Beatrice Portelli, Simone Scaboro, Enrico Santus, Hooman Sedghamiz, Emmanuele Chersoni, Giuseppe Serra
| Challenge: | Medical term normalization is a task of mapping a text to a large number of output classes. |
| Approach: | They propose a learning strategy that leverages hierarchical information to enhance generalizability of models. |
| Outcome: | The proposed strategy produces state-of-the-art performance on seen concepts and consistent improvements on unseen ones, allowing efficient zero-shot knowledge transfer across text typologies and datasets. |
ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Parity LLM Data Valuation (2025.naacl-long)
Copied to clipboard
Yanzhou Pan, Huawei Lin, Yide Ran, Jiamin Chen, Xiaodong Yu, Weijie Zhao, Denghui Zhang, Zhaozhuo Xu
| Challenge: | Large Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance. |
| Approach: | They propose a third-party data valuation approach that assesses the value of individual data samples and proposes a learning strategy to approximate LinFiK. |
| Outcome: | The proposed approach surpasses baselines in effectiveness and efficiency, showing significant scalability advantages as LLM parameters increase. |
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on curriculum learning rely on task-specific expertise and cannot generalize to different tasks. |
| Approach: | They propose to do in-sample curriculum learning for natural language generation tasks using human-crafted rules and a numeric score for each sample based on domain expertise to rank the model. |
| Outcome: | The proposed learning strategy generalizes well to different tasks and achieves significant improvements over baselines. |
Learning from Children: Improving Image-Caption Pretraining via Curriculum (2023.findings-acl)
Copied to clipboard
| Challenge: | Image-caption pretraining is a difficult problem as it requires multiple concepts (nouns) from captions to be aligned to multiple objects in images. |
| Approach: | They propose a curriculum learning framework that uses images to align multiple concepts to multiple objects in an image. |
| Outcome: | The proposed learning framework improves over pretraining from scratch, using a pretrained image or/and text encoder, low data regime etc. |
Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | auxiliary captions are generic and indistinguishable across visually similar videos . conventional captioning approaches are evaluated using language relevance scores . |
| Approach: | They propose a retrieval framework that directly optimizes caption generation using retrieval relevance scores. |
| Outcome: | The proposed retrieval framework optimizes caption generation using retrieval relevance scores . dual-group direct preference optimization is a learning strategy that supervises captioning . |
To Learn or Not to Learn: Replaced Token Detection for Learning the Meaning of Negation (2024.lrec-main)
Copied to clipboard
| Challenge: | State-of-the-art language models perform well on a variety of language tasks, but struggle with understanding negation cues in tasks like natural language inference (NLI). |
| Approach: | They propose a new learning strategy for negation building on ELECTRA’s replaced token detection objective. |
| Outcome: | The proposed approach leads to substantial gains on a variant of RTE with additional negation. |